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Introduction 


What is PISA? 

The Program for International Student Assessment (PISA) is a system of international 
assessments that allows countries to compare outcomes of learning as students near the 
end of compulsory schooling. PISA core assessments measure the performance of 15-year- 
old students in science, reading, and mathematics literacy every 3 years. Coordinated by 
the Organization for Economic Cooperation and Development (OECD), PISA was first 
implemented in 2000 in 32 countries. It has since grown to 73 education systems in 2015.’ 
The United States has participated in every cycle of PISA since its inception in 2000. 

In 2015, Massachusetts, North Carolina, and Puerto Rico also participated separately 
from the nation. Of these three, Massachusetts previously participated in PISA 2012. 

What PISA Measures 

PISAs goal is to assess students’ preparation for the challenges of life as young adults. 

The study assesses the application of knowledge in science, reading, and mathematics literacy 
to problems within a real-life context (OECD 1 999) . PISA does not focus explicitly on 
curricular outcomes and uses the term “literacy” in each subject area to indicate its broad 
focus on the application of knowledge and skills learned both in and outside of school. 

For example, when assessing science, PISA examines how well 15 -year-old students can 
understand, use, and reflect on science for a variety of real-life problems and settings that 
they may encounter in and out of school. 

Each PISA data collection cycle assesses one of the three core subject areas in depth 
(considered the major subject area), although all three core subjects are assessed in each 
cycle (the other two subjects are considered minor subject areas for that assessment year). 
Assessing all three subjects every 3 years allows countries to have a consistent source of 
achievement data in each of the three subjects while rotating one area as the primary focus 
over the years. Each subject is a major area once every three cycles. Science was the major 
subject area in 2015, as it was in 2006. In 2015, science, reading, and mathematics literacy 
were assessed through a computer-based assessment in the majority of countries, including 
the United States. 

In addition to these core assessments, education systems could participate in two optional 
assessment components: collaborative problem solving and financial literacy. The United 
States, Massachusetts, and North Carolina administered the computer-based assessment 
with both optional assessments while Puerto Rico administered the paper-based core 
assessment without the optional assessments. This report addresses results only of the science. 


' Of the 73 education systems that participated in PiSA2015, resuits for three of these — Argentina, Kazakhstan, and Maiaysia — are not 
included due to technical issues with their sampies that prevent results from being discussed in this report. For more information, see 
OECD (2016d). 
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mathematics, and reading literacy assessments. Results of the collahorative problem solving 
and hnancial literacy assessments will he released separately at a later date. 

PISA includes questionnaires that provide contextual information for interpreting 
student achievement. In 2015, the United States administered student, school, and 
teacher questionnaires. Students answered questions about their background, attitudes 
towards science, and learning strategies, among other topics. The principal of each 
participating school completed a school questionnaire that provided information on the 
school s demographics and learning environment. Lastly, up to 10 science teachers and 1 5 
nonscience teachers per school completed questionnaires on teaching practices, beliefs about 
teaching, and their qualihcations and backgrounds. Massachusetts and North Carolina 
administered all three questionnaires while Puerto Rico administered the student and school 
questionnaires only. 

Visit http://nces.ed.gov/surveys/pisa for more information on the PISA assessments, 
including information on how the assessments were designed and examples of PISA test 
items and questionnaires. PISA questions from the current and previous cycles can also 
be found at https://www.oecd.org/pisa/pisaproducts/pisa-test-questions.htm. 

Science Literacy 

In PISA 2015, the major subject was science literacy, defined as 

the ability to engage with science-related issues, and with the ideas of science, as a reflective 
citizen. A scientifically literate person, therefore, is willing to engage in reasoned discourse 
about science and technology which requires the competencies to: 

1. Explain phenomena scientifically: Recognize, offer, and evaluate explanations 
for a range of natural and technological phenomena. 

2. Evaluate and design scientific inquiry: Describe and appraise scientific investigations 
and propose ways of addressing questions scientifically. 

3. Interpret data and evidence scientifically: Analyze and evaluate data, claims and 
arguments in a variety of representations and draw appropriate scientific conclusions 
(OECD 2016a, p. 7). 
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More specifically, the PISA science assessment measures three science content categories 
and three science process categories: 

Science content categories (OECD 2016a, p. 18): 

• Physical systems: Are students able to understand the structures, properties, and chemical 
changes of matter, motion and forces, and energy and interactions between energy 

and matter? 

• Living systems: Do students have knowledge of cells, the concept of an organism, human 
systems, populations, ecosystems, and biospheres? 

• Earth and space systems: Are 15-year-olds able to comprehend the history and scale 
of the Universe and the Earth system’s structures, energy, and changes? 

Science process categories (OECD 2016a, pp. 8-16): 

• Explain phenomena scientifically: Can 15-year-olds recall appropriate scientihc 
knowledge, identify explanatory models, make and justify predictions and hypotheses, 
and explain the potential implications of scientific knowledge for society? 

• Evaluate and design scientific enquiry: Can students identify the questions explored in 

a given scientific study, propose and evaluate ways of exploring such questions, and describe 
and evaluate the methods scientists use to ensure data quality? 

• Interpret data and evidence scientifically: Are 15-year-olds able to analyze and interpret 
data to draw appropriate conclusions, identify the assumptions in science-related texts, 
recognize when arguments are based on scientific evidence and theory, and evaluate 
scientific arguments from different sources? 

Science literacy is reported in terms of an overall scale score and seven subscale scores 
(all of which are on a scale of 0-1,000).^ For the overall science literacy scale, results are 
reported also in terms of the percentage of students performing at each of seven proficiency 
levels. Exhibit 1 describes the seven science literacy proficiency levels and the cut scores 
associated with each proficiency level. 


^ Science literacy subscale score results by content and process categories can be found online at httD://nces.ed.aov/survevs/pisa/ 
Disa2015/index.aSD . 
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Exhibit 1 . Description of PISA proficiency levels on the science literacy scale 
2015 


Proficiency 
ievei and iower 
cut score 

Task descriptions 

Level 6 

708 

At Level 6, students can draw on a range of interrelated scientific ideas and concepts 
from the physical, life and earth and space sciences and use content, procedural 
and epistemic knowledge in order to offer explanatory hypotheses of novel scientific 
phenomena, events, and processes or to make predictions. In interpreting data and 
evidence, they are able to discriminate between relevant and irrelevant information and 
can draw on knowledge external to the normal school curriculum. They can distinguish 
between arguments that are based on scientific evidence and theory and those based 
on other considerations. Level 6 students can evaluate competing designs of complex 
experiments, field studies or simulations and justify their choices. 

Level 5 

633 

At Level 5, students can use abstract scientific ideas or concepts to explain unfamiliar 
and more complex phenomena, events, and processes involving multiple causal 
links. They are able to apply more sophisticated epistemic knowledge to evaluate 
alternative experimental designs and justify their choices and use theoretical knowledge 
to interpret information or make predictions. Level 5 students can evaluate ways of 
exploring a given question scientifically and identify limitations in interpretations of data 
sets including sources and the effects of uncertainty in scientific data. 

Level 4 

559 

At Level 4, students can use more complex or more abstract content knowledge, which is 
either provided or recalled, to construct explanations of more complex or less familiar 
events and processes. They can conduct experiments involving two or more independent 
variables in a constrained context. They are able to justify an experimental design, drawing 
on elements of procedural and epistemic knowledge. Level 4 students can interpret data 
drawn from a moderately complex data set or less familiar context, draw appropriate 
conclusions that go beyond the data, and provide justifications for their choices. 

Level 3 

484 

At Level 3, students can draw upon moderately complex content knowledge to identify 
or construct explanations of familiar phenomena. In less familiar or more complex 
situations, they can construct explanations with relevant cueing or support. They 
can draw on elements of procedural or epistemic knowledge to carry out a simple 
experiment in a constrained context. Level 3 students are able to distinguish between 
scientific and non-scientific issues and identify the evidence supporting a scientific claim. 

Level 2 

410 

At Level 2, students are able to draw on everyday content knowledge and basic 
procedural knowledge to identify an appropriate scientific explanation, interpret data, 
and identify the question being addressed in a simple experimental design. They can 
use basic or everyday scientific knowledge to identify a valid conclusion from a simple 
data set. Level 2 students demonstrate basic epistemic knowledge by being able to 
identify questions that could be investigated scientifically. 

Level la 

335 

At Level la, students are able to use basic or everyday content and procedural 
knowledge to recognize or identify explanations of simple scientific phenomenon. 

With support, they can undertake structured scientific enquiries with no more than two 
variables. They are able to identify simple causal or correlational relationships and 
interpret graphical and visual data that require a low level of cognitive demand. Level 
la students can select the best scientific explanation for given data in familiar personal, 
local and global contexts. 

Level 1b 

261 

At Level 1b, students can use basic or everyday scientific knowledge to recognize 
aspects of familiar or simple phenomenon. They are able to identify simple patterns 
in data, recognize basic scientific terms and follow explicit instructions to carry out a 
scientific procedure. 


NOTE: To reach a particular proficiency level, a student must correctly answer a majority of items at that level. Students were classified into 
science literacy levels according to their scores. Cut scores in the exhibit are rounded; exact cut scores can be found at http://nces.ed.aov/ 
survevs/pisa/pisa201 5/index. asp . Scores are reported on a scale from 0 to 1,000. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment (PISA), 2015. 
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Reading Literacy 

In PISA 2015, reading literacy is defined as 

understanding, using, reflecting on and engaging with written texts, in order to achieve 
one’s goals, to develop one’s knowledge and potential and to participate in society 
(OECD 2016b, p. 9). 

Reading literacy is also reported both in terms of proficiency levels and an overall scale score 
(on a scale of 0-1,000). Exhibit 2 describes the seven reading literacy proficiency levels and 
their respective cut scores. Since reading literacy was a minor domain in 2015, no subscale 
scores are reported.^ 


^ For readers interested in reading literacy subscale scores produced in previous cycies, please see the PISA 2009 results 
at httD://nces.ed.aov/survevs/Disa/ . 
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Exhibit 2. Description of PISA proficiency levels on reading literacy scale: 2015 


Proficiency 
ievei and iower 
cut score 

Task descriptions 

Level 6 

698 

At level 6, tasks typically require the reader to make multiple inferences, comparisons, and contrasts that 
are both detailed and precise. They require demonstration of a full and detailed understanding of one 
or more texts and may involve integrating information from more than one text. Tasks may require the 
reader to deal with unfamiliar ideas, in the presence of prominent competing information, and to generate 
abstract categories for interpretations. Reflect and evaluate tasks may require the reader to hypothesize 
about or critically evaluate a complex text on an unfamiliar topic, taking into account multiple criteria 
or perspectives, and applying sophisticated understandings from beyond the text. A salient condition 
for access and retrieve tasks at this level is precision of analysis and fine attention to detail that is 
inconspicuous in the texts. 

Level 5 

626 

At level 5, tasks that involve retrieving information require the reader to locate and organize several pieces 
of deeply embedded information, inferring which information in the text is relevant. Reflective tasks require 
critical evaluation or hypothesis, drawing on specialized knowledge. Both interpretative and reflective tasks 
require a full and detailed understanding of a text whose content or form is unfamiliar. For all aspects of 
reading, tasks at this level typically involve dealing with concepts that are contrary to expectations. 

Level 4 

553 

At level 4, tasks that involve retrieving information require the reader to locate and organize several 
pieces of embedded information. Some tasks at this level require interpreting the meaning of nuances of 
language in a section of text by taking into account the text as a whole. Other interpretative tasks require 
understanding and applying categories in an unfamiliar context. Reflective tasks at this level require 
readers to use formal or public knowledge to hypothesize about or critically evaluate a text. Readers must 
demonstrate an accurate understanding of long or complex texts whose content or form may be unfamiliar. 

Level 3 

480 

At level 3, tasks require the reader to locate, and in some cases recognize the relationship between, 
several pieces of information that must meet multiple conditions. Interpretative tasks at this level require 
the reader to integrate several parts of a text in order to identify a main idea, understand a relationship, or 
construe the meaning of a word or phrase. They need to take into account many features in comparing, 
contrasting or categorizing. Often the required information is not prominent or there is much competing 
information; or there are other text obstacles, such as ideas that are contrary to expectation or negatively 
worded. Reflective tasks at this level may require connections, comparisons, and explanations, or 
they may require the reader to evaluate a feature of the text. Some reflective tasks require readers to 
demonstrate a fine understanding of the text in relation to familiar, everyday knowledge. Other tasks do 
not require detailed text comprehension but require the reader to draw on less common knowledge. 

Level 2 

407 

At level 2, some tasks require the reader to locate one or more pieces of information, which may need 
to be inferred and may need to meet several conditions. Others require recognizing the main idea in 
a text, understanding relationships, or construing meaning within a limited part of the text when the 
information is not prominent and the reader must make low level inferences. Tasks at this level may 
involve comparisons or contrasts based on a single feature in the text. Typical reflective tasks at this level 
require readers to make a comparison or several connections between the text and outside knowledge, 
by drawing on personal experience and attitudes. 

Level 1a 

335 

At level la, tasks require the reader to locate one or more independent pieces of explicitly stated 
information; to recognize the main theme or author’s purpose in a text about a familiar topic, or to make 
a simple connection between information in the text and common, everyday knowledge. Typically, the 
required information in the text is prominent and there is little, if any, competing information. The reader is 
explicitly directed to consider relevant factors in the task and in the text. 

Level 1b 

262 

At level 1b, tasks require the reader to locate a single piece of explicitly stated information in a prominent 
position in a short, syntactically simple text with a familiar context and text type, such as a narrative or a 
simple list. The text typically provides support to the reader, such as repetition of information, pictures, or 
familiar symbols. There is minimal competing information. In tasks requiring interpretation the reader may 
need to make simple connections between adjacent pieces of information. 


NOTE: To reach a particular proficiency level, a student must correctly answer a majority of items at that level. Students were classified into reading literacy levels 
according to their scores. Cut scores in the exhibit are rounded; exact cut scores can be found at http://nces.ed.aov/survevs/pisa/pisa2015/index.asp . Scores are 
reported on a scale f om 0 to 1 ,000. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment (PISA), 2015 
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Mathematics Literacy 

In PISA 2015 mathematics literacy is defined as 


an individual’s capacity to formulate, employ, and interpret mathematics in a variety of contexts. 
It includes reasoning mathematically and using mathematical concepts, procedures, facts and 
tools to describe, explain and predict phenomena. It assists individuals to recognize the role that 
mathematics plays in the world and to make the well-founded judgments and decisions needed 
by constructive, engaged and reflective citizens (OECD 2016c, p. 5). 

As with other subject areas, mathematics literacy is reported both in terms of proficiency 
levels and an overall scale score (on a scale of 0-1,000). Exhibit 3 describes the six 
mathematics literacy proficiency levels and their respective cut scores. Since mathematics 
literacy was a minor subject in 2015, no subscale scores are reported.^ 


“ For readers interested in mathematics literacy subscale scores produced in previous cycies, please see the PISA 201 2 results 
at httD://nces.ed.aov/survevs/oisa/Disa2012/index.asD . 
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Exhibit 3. Description of PISA proficiency levels on the mathematics literacy 
scale: 2015 


Proficiency 
ievei and iower 
cut score 

Task descriptions 

Level 6 

669 

At level 6, students can conceptualize, generalize, and utilize information based on their 
investigations and modeling of complex problem situations, and can use their knowledge 
in relatively non-standard contexts. They can link different information sources and 
representations and flexibly translate among them. Students at this level are capable 
of advanced mathematical thinking and reasoning. These students can apply this 
insight and understanding, along with a mastery of symbolic and formal mathematical 
operations and relationships, to develop new approaches and strategies for attacking 
novel situations. Students at this level can reflect on their actions, and can formulate 
and precisely communicate their actions and reflections regarding their findings, 
interpretations, arguments and the appropriateness of these to the original situations. 

Level 5 

607 

At level 5, students can develop and work with models for complex situations, identifying 
constraints and specifying assumptions. They can select, compare, and evaluate 
appropriate problem-solving strategies for dealing with complex problems related to 
these models. Students at this level can work strategically using broad, well-developed 
thinking and reasoning skills, appropriate linked representations, symbolic and formal 
characterizations, and insight pertaining to these situations. They begin to reflect on their 
work and can formulate and communicate their interpretations and reasoning. 

Level 4 

545 

At level 4, students can work effectively with explicit models for complex concrete 
situations that may involve constraints or call for making assumptions. They can select 
and integrate different representations, including symbolic, linking them directly to 
aspects of real-world situations. Students at this level can utilize their limited range 
of skills and can reason with some insight, in straightforward contexts. They can 
construct and communicate explanations and arguments based on their interpretations, 
arguments, and actions. 

Level 3 

482 

At level 3, students can execute clearly described procedures, including those that require 
sequential decisions. Their interpretations are sufficiently sound to be a base for building 
a simple model or for selecting and applying simple problem-solving strategies. Students 
at this level can interpret and use representations based on different information sources 
and reason directly from them. They typically show some ability to handle percentages, 
fractions and decimal numbers, and to work with proportional relationships. Their 
solutions reflect that they have engaged in basic interpretation and reasoning. 

Level 2 

420 

At level 2, students can interpret and recognize situations in contexts that require no 
more than direct inference. They can extract relevant information from a single source 
and make use of a single representational mode. Students at this level can employ 
basic algorithms, formulae, procedures, or conventions to solve problems involving 
whole numbers. They are capable of making literal interpretations of the results. 

Level 1 

358 

At level 1 , students can answer questions involving familiar contexts where all relevant 
information is present and the questions are clearly defined. They are able to identify 
information and to carry out routine procedures according to direct instructions in 
explicit situations. They can perform actions that are almost always obvious and follow 
immediately from the given stimuli. 


NOTE: To reach a particular proficiency level, a student must correctly answer a majority of items at that level. Students were classified into 
mathematics literacy levels according to their scores. Cut scores in the exhibit are rounded; exact cut scores can be found at httoi/Znces. 
ed.aov/survevs/Disa/Disa201 5/index. asp . Scores are reported on a scale from 0 to 1,000. 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment (PISA), 2015. 
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Computer-Based Assessment 

In 2012, PISA began offering parts of the assessment through a computer-based system. 

For those countries, like the United States, that wanted to start the transition to a 
computer-based assessment, the 2012 assessment cycle included both paper-based and 
computer-based components. All sampled students took the paper-and-pencil based version 
of the reading, mathematics, and science literacy assessment. In addition, some countries 
(including the United States), chose to administer separate optional computer-based reading 
and mathematics literacy assessments to a subsample of students. For the 2015 cycle, all 
parts of the PISA assessment moved to a completely computer-based assessment platform 
(although some participating education systems chose to administer PISA in the traditional 
paper-and-pencil format) . 

The 2015 field trial included a mode effect study to determine whether or not the paper- 
based assessment and the fully computer-based assessment were psychometrically comparable 
and could be reported on the same scale. To do this, the mode effect study pooled data from 
all the participating education systems to examine whether differences in the delivery mode 
could be detected in the student outcomes. The study found few systematic differences 
between computer-based and paper-based student achievement at the international level. 
Although some differences between the computer-based and paper-based results were 
found among students who used computers infrequently or not at all, this group accounted 
for about 10 percent of students across countries. Overall, the mode effect portion of the 
field trial study found that paper-based items were comparable to their computer-based 
counterparts and the level of difficulty of items varied little between paper-based and 
computer-based modes (OECD 2016d; see also OECD forthcoming). This finding provided 
reasonable assurance that scores from prior cycles could be compared with those from the 
2015 cycle without the need for any adjustments or recalculations, and that scores derived 
from the paper-based version of PISA and the computer-based version can be reported on 
the same scale. 

for education systems that administered the computer-based assessment in the 2015 cycle, 
all parts of the study were computer-based, including the school, teacher, and student 
questionnaires. In 2015, some 57 out of 70 education systems, including the United States, 
chose to administer the computer-based assessment.^ Although Massachusetts and North 
Carolina also administered the computer-based assessment, Puerto Rico opted to administer 
the paper-based assessment. 


= A list of education systems that administered PISA 201 5 on computer and on paper is available in the OECD’s PISA 2015 International 
Report: Volume 1 (OECD 201 6d). 
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Reporting PISA 201 5 Results 

This report presents performance on PISA 2015 in science, reading, and mathematics literacy 
from a U.S. perspective. Results are presented for the 70 education systems that participated 
in PISA 2015 as well as Massachusetts, North Carolina, and Puerto Rico. The U.S. national 
results include both public and private schools. Massachusetts and North Carolina chose 
to sample public schools only. Puerto Rico sampled both public and private schools. 

To maintain the trend with previous administrations, PISA U.S. national results do not 
include Puerto Rico. 

In this report, results for each participating education system are presented in terms of 
average scale scores, percentile scores, and the percentage of 15-year-old students reaching 
selected proficiency levels, comparing the United States with other participating education 
systems. For percentile scores, the threshold (or cut) scores for the 1 0th percentile of the 
distribution of student performance (i.e., the bottom 10 percent of students) and the 90th 
percentile (i.e., the top 10 percent of students) are shown, providing a view of student 
performance that goes beyond the average score. The percentile ranges are specific to 
each education system’s distribution of scores, enabling users to compare cut scores across 
education systems. For proficiency levels, results are reported in terms of the percentage 
reaching levels 5 and above and the percentage below level 2. Higher proficiency levels 
represent the knowledge, skills, and capabilities needed to perform tasks of greater 
complexity. At levels 5 and 6, students demonstrate higher-level skills and may be referred 
to as “top performers” in the subject (see exhibits 1-3 for descriptions of the PISA proficiency 
levels). Conversely, students performing below level 2 are below what the OECD calls 
“a baseline level of proficiency that all young adults should be expected to attain in order to 
take advantage of further learning opportunities and participate fully in the social, economic 
and civic life of modern societies in a globalized world” (OECD 20l6d, p. 16).'’ This report 
also presents U.S. trends over time in science, reading, and mathematics literacy. Results for 
the collaborative problem solving and financial literacy assessments will be released in 2017. 

In reporting PISA results, the OECD differentiates between OECD member countries, 
of which there are 35, and all other participating education systems, some of which are 
countries and some of which are subnational entities (i.e., cities, states, provinces, and 
territories). In the OECD’s PISA reports, OECD member countries and other participating 
education systems are reported in the tables and figures in the main body of the report, along 
with the average for the OECD countries (the average of the 35 OECD member country 
averages with each country weighted equally) and are discussed in the accompanying text. 
Also, for some participating education systems, results for subnational entities — including, 
in 2015, Massachusetts, North Carolina, and Puerto Rico — are reported in appendices 
of the OECD PISA reports, but are not discussed in the report text. 


The data in this report focus on the highest- and lowest-level skills and knowledge along the science, reading, and mathematics literacy 
proficiency scales. For a full description of the percentage of students at every proficiency level, please see the data tables and figures 
at http://nces.ed.aov/survevs/pisa/pisa2015/index.asp . 
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To facilitate readers moving between the OECD and U.S. national PISA reports, this report’s 
tables and figures follow the OECD convention of placing OECD member countries and 
all other participating education systems in the main part of the tables and figures. These are 
all referred to as education systems in this report, of which there are 70 total. Massachusetts, 
North Carolina, and Puerto Rico are presented in a separate part of the tables and figures; 
results for these three education systems are discussed in the text but are not included in 
counts of education systems performing above, below, or not measurably different from the 
United States. 

This report is merely a first look at the PISA 2015 results and is by no means comprehensive. 
For in-depth results using PISA 2015 data, please see http://nces.ed.gov/surveys/pisa/ 
pisa20 15 /index, asp, which includes additional findings from the science, reading, and 
mathematics literacy assessments, science subscales, proficiency levels, percentile cut- 
scores, trends in performance, performance by students’ gender, race and ethnicity, and 
socioeconomic status, as well as more detailed results for Massachusetts, North Carolina, 
and Puerto Rico. Detailed notes on technical aspects of PISA 2015, including sample 
design, test design, and scoring, with an emphasis on the U.S. implementation, can 
be found on the NCES website at http://nces.ed.gov/surveys/pisa/pisa2015/index.asp. 

In addition, PISA results are available through the PISA International Data Explorer (IDE) 
at https://nces.ed.gov/surveys/pisa/idepisa/. The PISA IDE provides anyone with an internet 
connection the opportunity to create statistical tables and charts and conduct regression 
analyses. These analyses can be performed on student performance as well as contextual 
data, including student demographics, instructional experiences, and school characteristics, 
across all participating education systems. 

All statistically significant differences described in this report were tested at the .05 level 
of statistical significance. Differences that are statistically significant are discussed using 
comparative terms such as “higher” and “lower.” Differences that are not statistically 
significant are either not discussed or referred to as “not measurably different.” In almost 
all instances, the tests for significance used were standard t tests (see the technical notes at 
http://nces.ed.gOv/surveys/pisa/pisa2015/index.asp for additional details on interpreting 
statistical significance). No adjustments were made for multiple comparisons. 
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Selected Findings 


U.S. Performance in Science Literacy 

• In 2015, average scores of 15-year-olds in science literacy ranged from 556 in Singapore 
to 332 in the Dominican Republic. The U.S. average score was 496, which was not 
measurably different than the OECD average of 493. The U.S. average was lower than 
18 education systems, higher than 39, and not measurably different than 12 education 
systems. It was lower than the average in Massachusetts (529), not measurably different 
than in North Carolina (502), and higher than in Puerto Rico (403) (table 1). 

• Education systems varied in the range in performance between their highest performing 
students and their lowest performing students. The difference in science literacy scores 
between the highest performing (90th percentile) and low performing (10th percentile) 
students in the United States was 258 score points, which was not measurably different 
than the average difference across the OECD countries (247 score points) . It was smaller 
than in 4 education systems, not measurably different than in 26, and larger than in 39 
education systems. The U.S. score difference was not measurably different than the gap 
in Massachusetts (253) and North Carolina (252), and larger than the gap in Puerto Rico 
(226) (figure 1). 

• The distribution of students across the seven PISA science literacy proficiency levels in 
2015 showed that the percentages of top performing 15-year-old students (those scoring 
at proficiency levels 5 and above) ranged from 24 percent in Singapore to rounding 

to 0 percent in 10 education systems. Nearly one in ten U.S. 15 -year-olds (9 percent) 
scored at proficiency levels 5 and above, which was not measurably different than the 
OECD average (8 percent). The U.S. percentage of students at the top levels was lower 
than in 14 education systems, higher than in 34, and not measurably different than in 15 
education systems. The U.S. percentage at the top levels was lower than in Massachusetts 
(14 percent), and not measurably different than in North Carolina (9 percent) (figure 2; 
see also exhibit 1). 

• The percentages of low performing 15-year-old students (those scoring below proficiency 
level 2) — which is considered below the baseline of proficiency by the OECD — ranged 
from 6 percent in Vietnam to 86 percent in the Dominican Republic. In the United States, 
one in five 15-year-olds (20 percent) scored below level 2 in science literacy, which was 
not measurably different than the OECD average (2 1 percent) . The percentage of low 
performers in the United States was higher than in 2 1 education systems, lower than in 
37, and not measurably different than in 1 1 education systems. It was higher than the 
percentage in Massachusetts (12 percent), not measurably different than in North Carolina 
(18 percent), and lower than in Puerto Rico (55 percent) (figure 2; see also exhibit 1). 
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U.S. Performance in Reading Literacy 

• In 2015, the average scores of U.S. 15-year-olds in reading literacy was 497, and ranged 
among the other education systems from 535 in Singapore to 347 in Lebanon. The U.S. 
average was lower than the averages in 14 education systems, higher than in 42, and not 
measurably different than in 13 education systems and the OECD average. The overall 
U.S. average was lower than the average in Massachusetts (527), not measurably different 
than in North Carolina (500), and higher than in Puerto Rico (410) (table 2). 

• The distribution of student scores in reading literacy showed that the U.S. score difference 
between the 90th and 10th percentiles (259 points) was larger than the difference in 30 
education systems, not measurably different than in 30 other systems, and smaller than in 
9. The U.S. difference between the 90th and 10th percentiles was not measurably different 
than in Massachusetts (243), North Carolina (251), and Puerto Rico (250) (figure 3). 

• In reading literacy, the percentage of top performing students (those scoring at proficiency 
levels 5 and above) ranged from 1 8 percent in Singapore to rounding to 0 percent in 5 
education systems. One in ten U.S. 15-year-olds (10 percent) scored at proficiency levels 

5 and above in reading literacy in 2015. The percentage of U.S. top performers was higher 
than in more than half of the other educations systems (40 of 69), not measurably different 
than in 1 6 systems and the OECD average, and lower than in 8 systems. The percentage of 
top performers in the United States overall was lower than the percentage in Massachusetts 
(14 percent), not measurably different than in North Carolina (10 percent), and higher 
than in Puerto Rico (1 percent) (figure 4; see also exhibit 2). 

• The percentages of low performing 1 5-year-old students (those scoring below proficiency 
level 2) ranged from 9 percent in Hong Kong (China) to 79 percent in Algeria. Nearly 
one in five U.S. 15-year-olds (19 percent) scored below level 2, which was not measurably 
different than the OECD average (20 percent). This percentage was higher than in 14 
education systems, lower than in 36, and not measurably different than in 19 education 
systems. The percentage of low performers in in the United States overall was higher than 
the percentage in Massachusetts (11 percent), not measurably different than North Carolina 
(18 percent) and lower than in Puerto Rico (50 percent) (figure 4; see also exhibit 2). 

U.S. Performance in Mathematics Literacy 

• Average mathematics literacy scores in 2015 ranged from 564 in Singapore to 328 in the 
Dominican Republic, with the U.S. average score at 470. The U.S. average was lower than 
in more than half of the other education systems (36 of 69) as well as the OECD average, 
higher than in 28 education systems, and not measurably different than in 5. It was lower 
than the average in Massachusetts (500), not measurably different than in North Carolina 
(471), and higher than in Puerto Rico (378) (table 3). 
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• The difference in mathematics literacy scores between students at the 90th percentile 
of performance and those at the 10th percentile in the United States was 230 points, 
which was larger than the difference in 1 5 education systems, smaller than in 20, and not 
measurably different than in 34 education systems. The U.S. difference was not measurably 
different than the difference between the highest and lowest performers in Massachusetts 
(218) and North Carolina (225) and was larger than in Puerto Rico (197) (figure 5). 

• In 2015, the percentage of top performing students in mathematics literacy (those scoring 
at levels 5 and above) ranged from 35 percent in Singapore to rounding to 0 percent 

in five education systems. Six percent of U.S. 15-year-olds scored at proficiency levels 
5 and above, which was lower than the percentages in 36 educations and the OECD 
average, higher than in 24 education systems, and not measurably different than in 6 
systems. The overall U.S. percentage of top performers was lower than the percentage 
in Massachusetts (10 percent) and was not measurably different than in North Carolina 
(6 percent) (figure 6; see also exhibit 3). 

• The percentage of low performing students in PISA mathematics literacy (below 
proficiency level 2) ranged from 7 percent in Macau (China) to 9 1 percent in the 
Dominican Republic. In the United States, nearly three out of ten of 15-year-old students 
(29 percent) scored below proficiency level 2, which was higher than the OECD average 
of 23 percent. The percentage of U.S. low performers was higher than in more than half 
of the other education systems (35 of 69), lower than in 28 education systems, and not 
measurably different than in 6 systems. The percentage of low performers in mathematics 
literacy in the United States overall was larger than in Massachusetts (17 percent), was not 
measurably different than in North Carolina (29 percent), and smaller than in Puerto Rico 
(73 percent) (figure 6; see also exhibit 3). 

Eooking across student performance in science, reading, and mathematics literacy, 15 -year- 
olds in 1 1 education systems demonstrated higher average scores in all three subjects than 
students in the United States: Canada, Estonia, Einland, Germany, Hong Kong (China), 
Japan, Macau (China), New Zealand, Republic of Korea, Singapore, and Slovenia. 

In addition, 15 -year-olds in Massachusetts had higher average scores in all three subjects 
than 15-year-olds in the United States overall (tables 1, 2, and 3). 

U.S. Performance Over Time 

• The U.S. average scores in science and reading literacy in 2015 were not measurably 
different than the average scores observed in previous PISA assessment years with which 
comparisons can be made (2006, 2009, and 2012 for science literacy; 2000, 2003, 2009, 
and 2012 for reading literacy). The U.S. average score in mathematics literacy in 2015 was 
12 score points lower than the average score in 2012 and 18 score points lower than the 
average in 2009, but was not measurably different than the average mathematics literacy 
scores in 2003 and 2006 (table 4). 
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Figures and Tables 


Table 1. Average scores of 15-year-old students on the PISA 
science literacy scale, by education system: 2015 

Education system 

Averacpe score 

Education system Average 

score 

OECD average 

493 



Singapore 

556 0 

Iceland 

473® 

Japan 

538 0 

Israel 

467® 

Estonia 

534 0 

Malta 

465® 

Chinese Taipei 

532 0 

Slovak Republic 

461 ® 

Finland 

531 O 

Greece 

455® 

Macau (China) 

529 0 

Chile 

447® 

Canada 

528 0 

Bulgaria 

446® 

Vietnam 

525 0 

United Arab Emirates 

437® 

Hong Kong (China) 

523 0 

Uruguay 

435® 

B-S-J-G (China) 

5180 

Romania 

435® 

Korea, Republic of 

5160 

Cyprus 

433® 

New Zealand 

5130 

Moldova, Republic of 

428® 

Slovenia 

5130 

Albania 

427® 

Australia 

5100 

Turkey 

425® 

United Kingdom 

509 0 

Trinidad and Tobago 

425® 

Germany 

509 0 

Thailand 

421 ® 

Netherlands 

509 0 

Costa Rica 

420® 

Switzerland 

506 0 

Qatar 

418® 

Ireland 

503 

Colombia 

416® 

Belgium 

502 

Mexico 

416® 

Denmark 

502 

Montenegro, Republic of 

411 ® 

Poland 

501 

Georgia 

411 ® 

Portugal 

501 

Jordan 

409® 

Norway 

498 

Indonesia 

403® 

United States 

496 

Brazil 

401 ® 

Austria 

495 

Peru 

397® 

France 

495 

Lebanon 

386® 

Sweden 

493 

Tunisia 

386® 

Czech Republic 

493 

Macedonia, Republic of 

384® 

Spain 

493 

Kosovo 

378® 

Latvia 

490 

Algeria 

376® 

Russian Federation 

487® 

Dominican Republic 

332® 

Luxembourg 

483® 



Italy 

481 ® 



Hungary 

477® 

U.S. states and territories 


Lithuania 

475® 

Massachusetts 

529 0 

Croatia 

475® 

North Carolina 

502 

Buenos Aires (Argentina) 

475® 

Puerto Rico 

403® 


O Average score is higher than U.S. average score. 

® Average score is iower than U.S. average score. 

NOTE: Education systems are ordered by 2015 average score. The OECD average is the average of the national 
averages of the OECD member countries, with each country weighted equaliy. Scores are reported on a scaie from 
0 to 1 ,000. Ail average scores reported as higher or iower than the U.S. average score are different at the .05 ievei 
of statistical significance, itaiics indicate non-OECD countries and education systems. B-S-J-G (China) refers to the 
four PiSA participating China provinces: Beijing, Shanghai, Jiangsu, and Guangdong. Resuits for Massachusetts and 
North Caroiina are for pubiic school students only. Although Argentina, Malaysia, and Kazakhstan participated in PiSA 
2015, technicai probiems with their samples prevent results from being discussed in this report. The standard errors 
of the estimates are shown in table SI available at httD://nces.ed.aov/survevs/Disa/PISA2015/index.asD . 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student 
Assessment (PISA), 2015. 
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Figure 1. Average scores and 10th and 90th percentile cut scores of 15-year-old students 
on the PISA science literacy scale, by education system; 2015 
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* p<.05. Cut score gap is significantly different from the U.S. 90th to 10th percentile cut score gap at the .05 level of significance. 

NOTE: Education systems are ordered by cut score gap, from smaller to larger. This table shows the threshold (or cut) scores for the following: (a) 1 0th 
percentile — the bottom 10 percent of students; (b) 90th percentile — the top 10 percent of students. The cut score gap for each education system is the 
difference between its 90th and 10th percentile cut scores. The percentile ranges are specific to each education system’s distribution of scores, enabling users 
to compare cut scores across education systems. The OECD average is the average of the national averages of the OECD member countries, with each 
country weighted equally. Scores are reported on a scale from 0 to 1 ,000. Italics indicate non-OECD countries and education systems. B-S-J-G (China) refers 
to the four PISA participating China provinces: Beijing, Shanghai, Jiangsu, and Guangdong. Results for Massachusetts and North Carolina are for public school 
students only. Although Argentina, Malaysia, and Kazakhstan participated in PISA 201 5, technical problems with their samples prevent results from being 
discussed in this report. The standard errors of the estimates are shown in table Sib available at httD://nces.ed.aov/survevs/Disa/PISA2015/index.asD . 
SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment (PISA), 2015. 
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Figure 2. Percentage of 1 5-year-old students performing at PISA science literacy proficiency 
levels below level 2 and levels 5 and above, by education system: 2015 
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NOTE: Education systems are ordered by percentage of 1 5-year-olds in Levels 5 and above. To reach a particular proficiency level, students must correctly 
answer a majority of items at that level. Students were classified into science proficiency levels according to their scores. Cut scores for each proficiency level 
can be found in exhibit 1 of this report and in table A-1 available at http://nces.ed.aov/survevs/pisa/PISA2015/index.asp . The OECD average is the average of the 
national percentages of the OECD member countries, with each country weighted equally. Italics indicate non-OECD countries and education systems. B-S-J-G 
(China) refers to the four PISA participating China provinces: Beijing, Shanghai, Jiangsu, and Guangdong. Results for Massachusetts and North Carolina are for 
public school students only. Although Argentina, Malaysia, and Kazakhstan participated in PISA 2015, technical problems with their samples prevent results from 
being discussed in this report. The standard errors of the estimates are shown in table S2b available at http://nces.ed.aov/survevs/pisa/PISA2015/index.asp . 
SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment (PISA), 2015. 
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Table 2. Average scores of 15-year-old students on the PISA 
reading literacy scale, by education system: 2015 

Education system 

Average score 

Education system Average 

score 

OECD average 

493 



Singapore 

535 0 

Lithuania 

472® 

Hong Kong (China) 

527 0 

Hungary 

470® 

Canada 

527 0 

Greece 

467® 

Finland 

526 0 

Chile 

459® 

Ireland 

521 O 

Slovak Republic 

453® 

Estonia 

5190 

Maita 

447® 

Korea, Republic of 

5170 

Cyprus 

443® 

Japan 

5160 

Uruguay 

437® 

Norway 

5130 

Romania 

434® 

New Zealand 

509 0 

United Arab Emirates 

434® 

Germany 

509 0 

Bulgaria 

432® 

Macau (China) 

509 0 

Turkey 

428® 

Poland 

506 0 

Costa Rica 

427® 

Slovenia 

505 0 

Trinidad and Tobago 

427® 

Netherlands 

503 

Montenegro, Republic of 

427® 

Australia 

503 

Colombia 

425® 

Sweden 

500 

Mexico 

423® 

Denmark 

500 

Moidova, Republic of 

416® 

France 

499 

Thaiiand 

409® 

Belgium 

499 

Jordan 

408® 

Portugal 

498 

Brazil 

407® 

United Kingdom 

498 

Albania 

405® 

Chinese Taipei 

497 

Qatar 

402® 

United States 

497 

Georgia 

401 ® 

Spain 

496 

Peru 

398® 

Russian Federation 

495 

Indonesia 

397® 

B-S-J-G (China) 

494 

Tunisia 

361 ® 

Switzerland 

492 

Dominican Republic 

358® 

Latvia 

488® 

Macedonia, Republic of 

352® 

Czech Republic 

487® 

Algeria 

350® 

Croatia 

487® 

Kosovo 

347® 

Vietnam 

487® 

Lebanon 

347® 

Austria 

485® 



Italy 

485® 



Iceland 

482® 

U.S. states and territories 


Luxembourg 

481 ® 

Massachusetts 

527 0 

Israel 

479® 

North Caroiina 

500 

Buenos Aires (Argentina) 

475® 

Puerto Rico 

410® 


O Average score is higher than U.S. average score. 

® Average score is iower than U.S. average score. 

NOTE: Education systems are ordered by 2015 average score. The OECD average is the average of the national 
averages of the OECD member countries, with each country weighted equaliy. Scores are reported on a scaie from 
0 to 1 ,000. Ail average scores reported as higher or iower than the U.S. average score are different at the .05 ievei 
of statistical significance, itaiics indicate non-OECD countries and education systems. B-S-J-G (China) refers to the 
four PiSA participating China provinces: Beijing, Shanghai, Jiangsu, and Guangdong. Resuits for Massachusetts and 
North Caroiina are for pubiic school students only. Although Argentina, Malaysia, and Kazakhstan participated in PiSA 
2015, technicai probiems with their samples prevent results from being discussed in this report. The standard errors 
of the estimates are shown in table R1 available at httD://nces.ed.aov/survevs/Disa/PISA201 S/index.asp . 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student 
Assessment (PISA), 2015. 
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Figure 3. Average scores and 10th and 90th 
on the PISA reading literacy scale, 


percentile cut scores of 15-year-old students 
by education system: 2015 
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* p<.05. Cut score gap is significantly different from the U.S. 90th to 10th percentile cut score gap at the .05 level of significance. 

NOTE: Education systems are ordered by cut score gap, from smaller to larger. This table shows the threshold (or cut) scores for the following: (a) 10th 
percentile — the bottom 1 0 percent of students; (b) 90th percentile — the top 1 0 percent of students. The cut score gap for each education system is the 
difference between its 90th and 10th percentile cut scores. The percentile ranges are specific to each education system's distribution of scores, enabling users 
to compare cut scores across education systems. The OECD average is the average of the national averages of the OECD member countries, with each country 
weighted equally. Scores are reported on a scale from 0 to 1 ,000. Italics indicate non-OECD countries and education systems. B-S-J-G (China) refers to the four 
PISA participating China provinces: Beijing, Shanghai, Jiangsu, and Guangdong. Results for Massachusetts and North Carolina are for public school students 
only. Although Argentina, Malaysia, and Kazakhstan participated in PISA 2015, technical problems with their samples prevent results from being discussed in 
this report. The standard errors of the estimates are shown in table R1 b available at http ://nces.ed.aov/survevs/Disa/PlSA201 5/index. asp . 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment (PISA), 2015. 
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Figure 4. Percentage of 15-year-old students performing at PISA reading literacy proficiency 
levels below level 2 and levels 5 and above, by education system: 2015 
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! Interpret with caution. Estimate Is unstable due to high coefficient of variation (>30 percent and <=50 percent), 
t Reporting standards not met due to coefficient of variation over 50 percent. 

* p<.05. Significantly different from the U.S. percentage at the .05 level of significance. 

NOTE: Education systems are ordered by percentage of 1 5-year-olds in Levels 5 and above. To reach a particular proficiency level, students must correctly 
answer a majority of items at that level. Students were classified into reading proficiency levels according to their scores. Cut scores for each proficiency level 
can be found in exhibit 2 of this report and in table A-1 at http://nces.edu.gov/surveys/pisa/PISA201 5/index.asp. The OECD average is the average of the 
national percentages of the OECD member countries, with each country weighted equally. Italics indicate non-OECD countries and education systems. B-S-J-G 
(China) refers to the four PISA participating China provinces: Beijing, Shanghai, Jiangsu, and Guangdong. Results for Massachusetts and North Carolina are for 
public school students only. Although Argentina, Malaysia, and Kazakhstan participated in PISA 2015, technical problems with their samples prevent results from 
being discussed in this report. The standard errors of the estimates are shown in table R2b available at httD://nces.ed.aov/survevs/Disa/PISA2015/index.asD . 
SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment (PISA), 2015. 
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Table 3. Average scores of 15-year-old students on the PISA 

mathematics literacy scale, by education system: 2015 


Education system 

Average score 

Education system Average 

score 

OECD average 

490 0 



Singapore 

564 0 

Israel 

470 

Hong Kong (China) 

548 0 

United States 

470 

Macau (China) 

544 0 

Croatia 

464 

Chinese Taipei 
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Buenos Aires (Argentina) 
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Japan 

532 0 

Greece 

454® 

B-S-J-G (China) 

531 O 

Romania 

444® 

Korea, Republic of 

524 0 

Bulgaria 

441 ® 

Switzerland 

521 O 

Cyprus 
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United Arab Emirates 
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Canada 

5160 

Chile 

423® 

Netherlands 
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Turkey 

420® 

Denmark 

511 O 
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420® 

Finland 

511 O 
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418® 

Slovenia 

5100 

Montenegro, Republic of 

418® 

Belgium 

507 0 

Trinidad and Tobago 

417® 

Germany 

506 0 

Thaiiand 

415® 

Poland 

504 0 

Albania 

413® 

Ireland 

504 0 

Mexico 

408® 

Norway 

502 0 

Georgia 

404® 

Austria 

497 0 

Qatar 

402® 

New Zealand 

495 0 

Costa Rica 

400® 

Vietnam 

495 0 

Lebanon 

396® 

Russian Federation 
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Colombia 

390® 

Sweden 

494 0 

Peru 

387® 

Australia 

494 0 

Indonesia 

386® 

France 

493 0 

Jordan 

380® 

United Kingdom 

492 0 

Brazil 

377® 

Czech Republic 

492 0 

Macedonia, Repubiic of 

371 ® 

Portugal 

492 0 

Tunisia 

367® 

Italy 

490 0 

Kosovo 

362® 

Iceland 

488 0 

Algeria 

360® 

Spain 

486 0 

Dominican Republic 

328® 

Luxembourg 

486 0 



Latvia 

482 0 



Maita 

479 0 

U.S. states and territories 


Lithuania 

478 0 

Massachusetts 

500 0 

Hungary 

477 

North Caroiina 

471 

Slovak Republic 

475 

Puerto Rico 

378® 


O Average score is higher than U.S. average score. 

® Average score is iower than U.S. average score. 

NOTE: Education systems are ordered by 2015 average score. The OECD average is the average of the national 
averages of the OECD member countries, with each country weighted equaliy. Scores are reported on a scaie from 
0 to 1 ,000. Ail average scores reported as higher or iower than the U.S. average score are ditferent at the .05 ievei 
of statistical significance, itaiics indicate non-OECD countries and education systems. B-S-J-G (China) refers to the 
four PiSA participating China provinces: Beijing, Shanghai, Jiangsu, and Guangdong. Resuits for Massachusetts and 
North Caroiina are for pubiic school students only. Although Argentina, Malaysia, and Kazakhstan participated in PiSA 
2015, technicai probiems with their samples prevent results from being discussed in this report. The standard errors 
of the estimates are shown in table Ml available at httD://nces.ed.aov/survevs/Disa/PISA2015/index.asD . 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student 
Assessment (PISA), 2015. 
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Figure 5. Average scores and 10th and 90th percentile cut scores of 15-year-old students 
on the PISA mathematics literacy scale, by education system: 2015 
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* p<.05. Cut score gap is significantly different from the U.S. 90th to 10th percentile cut score gap at the .05 level of significance. 

NOTE: Education systems are ordered by cut score gap, from smaller to larger. This table shows the threshold (or cut) scores for the following: (a) 1 0th 
percentile — the bottom 10 percent of students; (b) 90th percentile — the top 10 percent of students. The cut score gap for each education system is the 
difference between its 90th and 10th percentile cut scores. The percentile ranges are specific to each education system’s distribution of scores, enabling users 
to compare cut scores across education systems. The OECD average is the average of the national averages of the OECD member countries, with each 
country weighted equally. Scores are reported on a scale from 0 to 1 ,000. Italics indicate non-OECD countries and education systems. B-S-J-G (China) refers 
to the four PISA participating China provinces: Beijing, Shanghai, Jiangsu, and Guangdong. Results for Massachusetts and North Carolina are for public school 
students only. Although Argentina, Malaysia, and Kazakhstan participated in PISA 201 5, technical problems with their samples prevent results from being 
discussed in this report. The standard errors of the estimates are shown in table Ml b available at httD://nces.ed.aov/survevs/pisa/PISA2015/index.aSD . 
SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment (PISA), 2015. 
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Figure 6. Percentage of 15-year-old students performing at PISA mathematics literacy 

proficiency levels below level 2 and levels 5 and above, by education system: 2015 
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t Reporting standards not met due to coefficient of variation over 50 percent. 

* p<.05. Significantly different from the U.S. percentage at the .05 level of significance. 

NOTE: Education systems are ordered by percentage of 15-year-olds in Levels 5 and above. To reach a particular proficiency level, students must correctly 
answer a majority of items at that level. Students were classified into mathematics proficiency levels according to their scores. Cut scores for each proficiency 
level can be found in exhibit 3 of this report and in table A-1 at http://nces.edu.gov/sun/eys/pisa/PISA2015/index.asp. The OECD average is the average of the 
national percentages of the OECD member countries, with each country weighted equally. Italics indicate non-OECD countries and education systems. B-S-J-G 
(China) refers to the four PISA participating China provinces: Beijing, Shanghai, Jiangsu, and Guangdong. Results for Massachusetts and North Carolina are for 
public school students only. Although Argentina, Malaysia, and Kazakhstan participated in PISA 2015, technical problems with their samples prevent results from 
being discussed in this report. The standard errors of the estimates are shown in table M2b available at http://nces.ed.aov/survevs/Disa/PISA2015/index.aSD . 
SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment (PISA), 2015. 
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Table 4. Average scores and changes in average scores of U.S. 15-year-old 
students on the PISA science, reading, and mathematics literacy 
scales: 2003, 2006, 2009, 2012, and 2015 


Average score Change in average score 


Subject 

2003 

2006 

2009 

2012 

2015 

2015-2003 

2015-2006 

2015-2009 

2015-2012 

Science 

t 

489 

502 

497 

496 

t 

O 

O 

O 

Reading 

t 

t 

500 

498 

497 

t 

t 

o 

o 

Mathematics 

483 

474 

487 

481 

470 

O 

o 

® 

® 


O Average score in 201 5 is higher than the average score in the comparison year at the .05 levei of statisticai significance. 

OAverage score in 2015 is not measurabiy ditferent from average score in comparison year. 

® Average score in 201 5 is iower than the average score in the comparison year at the .05 levei of statistical significance, 
t Not applicable. Although science was assessed in 2000 and 2003, because the science framework was revised for 2006, it is possible to 
look at changes in science only from 2006 forward. Similarly, although reading was assessed in 2000, 2003, and 2006, and mathematics was 
assessed in 2000, because the reading framework was revised for PISA 2009 and mathematics framework was revised for PISA 2003, it is 
possible to look at changes in reading only from 2009 forward and in mathematics only from 2003 forward. 

NOTE: All average scores reported as higher or lower than the comparison year are ditferent at the .05 level of statistical significance. 

For information on the results for these three countries, see OECD (201 6d). The standard errors of the estimates are shown in table T1 
available at httD://nces.ed.aov/survevs/Disa/PISA2015/index.asr) . 

SOURCE: Organization for Economic Cooperation and Development (OECD), Program for International Student Assessment (PISA), 2003, 
2006, 2009, 2012, and 2015. 
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Technical Notes 


This section briefly describes features of the PISA 2015 assessment, with a particular focus on 
implementation in the United States. For further details about the assessment and any of the 
topics discussed here, see the fuller description of technical notes on the NCES PISA website 
at http://nces.ed.gOv/surveys/pisa/pisa2015/index.asp as well as the OECD’s PISA 2015 
Technical Report (forthcoming). 

Sampling and Response Rates 

The OECD required all participating education systems to adhere to the PISA 2015 
technical standards (OECD forthcoming), which provided detailed specifications for the 
required target population, sampling, response rates, translation, assessment administration, 
and data submission. According to the standards, the international desired population 
in each education system consisted of 15-year-olds attending either publicly or privately 
controlled schools in grade 7 and higher. More specifically, the technical standards required 
that students in the sample be 1 5 years and 3 months to 1 6 years and 2 months at the 
beginning of the testing period (referred to as “1 5-year-olds” or “1 5-year-old students”). 

The maximum length of the testing period was 42 consecutive days. Most education systems 
conducted testing from March through August 2015.' Tire sample design for PISA 2015 
was a stratified systematic sample, with sampling probabilities proportional to the estimated 
number of 15-year-old students in the school based on grade enrollments. 

The U.S. PISA 2015 national school sample consisted of 240 schools, which was higher 
than the international sampling minimum of 150 to offset anticipated school nonresponse 
and ineligibility. The samples for Massachusetts and North Carolina consisted of 59 schools 
each; the sample for Puerto Rico consisted of 55 schools. TTie U.S. national and Puerto Rico 
samples included both public and private schools; the Massachusetts and North Carolina 
samples included public schools only. As with the PISA national sample, the samples for 
Massachusetts, North Carolina, and Puerto Rico were also increased from the international 
minimum of 50 schools for subnational entities to offset anticipated school nonresponse and 
ineligibility. In the U.S. national, Massachusetts, and North Carolina samples, 42 students 
in each school took the core PISA assessment in mathematics literacy, science literacy, reading 
literacy, and collaborative problem solving. Eleven of the 42 students in the U.S. national, 
Massachusetts, and North Carolina samples were also subsampled to take the financial 
literacy assessment, which was held in a separate session after the main PISA assessment. 

The financial literacy assessment was not administered to the 42 students in each school 
in Puerto Rico. 


’ The United States and the United Kingdom were given permission to move the testing dates to September through November in an effort 
to improve response rates. 


29 


A total of 177 schools participated in the administration of narional PISA, including 142 
participating schools sampled as part of the original sample and 35 schools sampled as 
replacements for nonparticipating “original” schools. The overall weighted school response 
rate after replacements was 83 percent. For the United States as a whole, the weighted 
student response rate was 90 percent and the student exclusion rate was 3 percent. 

In Massachusetts, there were 48 participating schools (out of 53 eligible schools), resulting 
in an overall weighted school response rate of 92 percent. The overall weighted student 
response rate was 90 percent and the overall student exclusion rate was 4 percent. All eligible 
schools (54) participated in North Carolina, yielding an overall weighted school response 
rate of 1 00 percent with an overall weighted student response rate of 92 percent and an 
exclusion rate of 5 percent. All of Puerto Rico’s 47 eligible schools participated (100 percent). 
The weighted student response and exclusion rates in Puerto Rico were 93 percent and 
3 percent, respectively. 

See http://nces.ed.gOv/surveys/pisa/pisa2015/index.asp for PISA international sampling 
guidelines and requirements regarding accommodations, exclusions, and response rate 
requirements, as well as response rates of all participating education systems. 

Assessment and Questionnaires 

The 2015 assessment instruments were developed by international experts and PISA 
international consortium test developers and included items submitted by participating 
education systems. In addition to the core subject areas of science, mathematics and reading 
literacy, PISA also offered two optional assessment components: financial literacy and 
collaborative problem solving. The United States administered both the core subject areas 
and the two optional assessment components. Massachusetts, North Carolina, and Puerto 
Rico administered the core PISA assessment components. In addition, Massachusetts 
and North Carolina administered the financial literacy and collaborative problem solving 
components. 

All mathematics and reading items in the 2015 assessment instrument were trend items from 
previous assessments. Science items included both trend items and new items developed for 
2015. Items were reviewed by representatives of each country for possible bias and relevance 
to PISAs goals and by PISA subject-matter expert groups. To further examine potential biases 
and design issues in the PISA assessment, all participating education systems field tested the 
assessment items in spring 2014. After the field trial, items that did not meet the established 
measurement criteria or were otherwise found to include intrinsic biases were dropped from 
the main assessment. 

The final 2015 main study computer-based assessment included six clusters from each of 
the trend domains of science, reading, and mathematics literacy, six clusters of new science 
literacy test items, and three clusters of new collaborative problem solving materials. The 
clusters were allocated in a rotated design to create six groups of test forms. Every student 
taking the assessment answered science items, and at least one but up to two of the other 
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subjects of mathematics literacy, reading literacy, and/or collaborative problem solving. 
Students who were subsampled for the financial literacy assessment returned for a second 
session in which the focus was only on financial literacy and the accompanying student 
questionnaire. For education systems administering the paper-based version of PISA, as in 
the case of Puerto Rico, the assessment included six clusters from each of the trend domains 
of science, reading, and mathematics literacy only. Every student taking the paper-based 
assessment answered science items, but not all students answered mathematics literacy 
or reading literacy items. In order to keep PISA as inclusive as possible and to keep the 
exclusion rate down, the United States, Massachusetts, and North Carolina used the UH 
(‘ Une Heure) instrument designed for students with special education needs (Puerto Rico 
did not use the UH instrument) . 

Approximately 65 percent of science items were multiple-choice and 35 percent of science 
items were open response. For reading and mathematics items, approximately 40 percent 
were multiple choice and 60 percent open response. Open response items were graded by 
trained scorers following international coding guidelines. 

After the cognitive assessment, students also completed a questionnaire designed to provide 
information about their backgrounds, attitudes, and experiences in school. Principals in 
schools where PISA was administered also completed a questionnaire designed to provide 
information on their school’s structure, resources, instruction, climate, and policies. In 
addition, in the U.S. national school sample, Massachusetts school sample, and North 
Carolina school sample, a sample of teachers within each school were selected to complete a 
computer-based questionnaire. (Puerto Rico did not administer the teacher questionnaire.) 

See http://nces.ed.gOv/surveys/pisa/pisa2015/index.asp for more information about the PISA 
2015 assessment design and questionnaires. 

Reporting Results 

In PISA, results are generally reported in two ways: scale scores (on a scale of 0 to 1 ,000) and 
the percentage of students reaching each proficiency level. Scale scores are reported in terms 
of both average scores and the threshold (or cut) scores at certain percentiles. In addition to 
scale scores as the basic form of measurement, PISA describes student performance in terms 
of levels of proficiency. Higher levels represent the knowledge, skills, and capabilities needed 
to perform tasks of increasing complexity. Proficiency level results are reported in terms of 
percentages of the student population at each of the predefined levels. In some displays, 
comparisons are made to the OECD average, which is the average of the 35 OECD member 
country averages with each country weighted equally. See http://nces.ed.gov/surveys/pisa/ 
pisa2015/index.asp for more information about PISA scaling and proficiency levels. 
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Sampling Weights and Standard Errors 

The use of sampling weights is necessary to compute statistically sound estimates. Adjusted 
survey weights adjust for the probabilities of selection for individual schools and students, 
for school or student nonresponse, and for errors in estimating the size of the school or the 
number of 15 -year-olds in the school at the time of sampling. 

As with any study, there are limitations to PISA 2015 that should be taken into 
consideration. Estimates produced using data from PISA 2015 are subject to two types 
of error: nonsampling errors and sampling errors. The sources of nonsampling errors are 
typically problems such as unit and item nonresponse, the differences in respondents’ 
interpretations of the meaning of survey questions, and mistakes in data preparation. 
Sampling errors arise when a sample of the population, rather than the whole population, 
is used to estimate some statistic. Different samples from the same population would likely 
produce somewhat different estimates of the statistic in question. This uncertainty is referred 
to as sampling variance and is usually expressed as the standard error of a statistic estimated 
from sample data. Standard errors for all statistics reported in this report are available online 
at http://nces.ed.gOv/surveys/pisa/pisa2015/index.asp. 

Statistical Comparisons 

Comparisons made in this report have been tested for statistical significance. For 
example, in the commonly made comparison of OECD averages to U.S. averages, tests 
of statistical significance were used to establish whether or not the observed differences 
from the U.S. average were statistically significant. In almost all instances, the tests for 
significance used were standard t tests. These fell into three categories according to the 
nature of the comparison being made: comparisons of independent samples, comparisons 
of nonindependent samples, and comparisons of performance over time. A difference is 
“significant” if the probability associated with the t test is less than .05. If a test is significant 
this implies that difference in the observed means in the sample represents a real difference 
in the population. When analyzing change in performance over time, the test for significance 
includes the addition of a standard error term. This is because the uncertainty that results 
from link item sampling, referred to as a “linking error,” must be taken into account when 
making certain comparisons between previous rounds of PISA (2003, 2006, 2009, and 
2012) and PISA 2015 results. No adjustments were made for multiple comparisons. 

More detailed information on the implementation of PISA in the United States is available 
on the NCES website at http://nces.ed.gov/surveys/pisa/pisa2015/index.asp. 


32 


This page intentionally left blank 




